Solana: A new architecture for a high 
performance blockchain v0.8.14 


Anatoly Yakovenko 
anatoly@solana.io 


Legal Disclaimer Nothing in this White Paper is an offer to sell, or the solicitation of an offer 
to buy, any tokens. Solana is publishing this White Paper solely to receive feedback and comments from 
the public. If and when Solana offers for sale any tokens (or a Simple Agreement for Future Tokens), it 
will do so through definitive offering documents, including a disclosure document and risk factors. Those 
definitive documents also are expected to include an updated version of this White Paper, which may 
differ significantly from the current version. If and when Solana makes such an offering in the United 
States, the offering likely will be available solely to accredited investors. 

Nothing in this White Paper should be treated or read as a guarantee or promise of how Solana’s 
business or the tokens will develop or of the utility or value of the tokens. This White Paper outlines 
current plans, which could change at its discretion, and the success of which will depend on many factors 
outside Solana’s control, including market-based factors and factors within the data and cryptocurrency 
industries, among others. Any statements about future events are based solely on Solana’s analysis of the 
issues described in this White Paper. That analysis may prove to be incorrect. 


Abstract 


This paper proposes a new blockchain architecture based on Proof ` 


of History (PoH) - a proof for verifying order and passage of time 


. PoH is used to encode trustlessypassagejof timesinto 
When used alongside a 


consensus algorithm such as Proofiof WorksPoW)corProofiof:Staken 
(PoS), PoH can redueeunessdeing overhead inlalByzanitinellaulilol 


erant replicated state machine, resulting in sub-second finality times. 
This paper also proposes two algorithms that leverage the time keep- 
ing properties of the PoH ledger - a POS algorithm that can recover 
from partitions of any size and an efficient streaming Proof of Replica- 
tion (PoRep)» The combination of PoRep and PoH provides a defense 
against forgery of the ledger with respect to time (ordering) and stor- 
age. The protocol is analyzed on a 1 gbps network, and this paper 
shows that throughput up to 710k transactions per second is possible 
with today’s hardware. 


1 Introduction 


Blockchain is an implementation of a fault tolerant replicated state machine. 
Current publicly available blockchains do not rely on time, or make a weak- 
assumption about the participant’s abilities to keep time [4, 5]. Each node 


in the network usually relies on their own local clock without knowledge of 
any other participants clocks in the network. The lack of a trusted source 


of time means that when a message timestamp is used to accept or rejecta 
will make the exact same choice? The PoH presented here is designed to 


create a ledger with verifiable passage of time, i.e. duration between events 
and message ordering. It is anticipated that every node in the network will 
be able to rely on the recorded passage of time in the ledger without trust. 


2 Outline 


The remainder of this article is organized as follows. Overall system design is 
described in Section 3. In depth description of Proof of History is described 
in Section 4. In depth description of the proposed Proof of Stake consensus 
algorithm is described in Section 5. In depth description of the proposed 
fast Proof of Replication is described in Section 6. System Architecture and 
performance limits are analyzed in Section 7. A high performance GPU 
friendly smart contracts engine is described in Section 7.5 


3 Network Design 


As shown in Figure 1, at any given time a system node is designated as 
Leader to generate a Proof of History sequence, l 
readyconsistencyrandranverifiableipassagerofitime. The Leader 

messages and orders them such that they can be efficiently processed by other 
nodes in the system, maximizingsthroughput. It «xecitesitheltransactions 
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Figure 1: Transaction flow throughout the network. 


In a non partitioned !state, at any given time,ttheretistone leader in the 
network. Each Verifié? node has the same hardware capabilities as a Leader 
and calbélélécted as a Leader) this is done through Pos based elections. 
Elections for the proposed PoS algorithm are covered in depth in Section 5.6. 

In terms of CAP theorem, Consisteneyisalmostalways picked over Avail- 

ility i In case of a large partition, this paper 
proposes a mechanism to recover control of the network from a partition of 
any size. This is covered in depth in Section 5.12. 


4 Proof of History 


Proof of History is a sequence of computation that can provide a way to 
cryptographically verify passage of time between two events. It uses a eryp-— 
tographically secure function written so that output cannot be predicted — 
from the input, and must be completely executed to generate the output. 
The function is rim in a sequence on a Single Cote, itsyprevioussoutputyas 


the current input, periodically recording the current output, and how many 
times it’s been called. The output can then be re-computed and verified 


by external computers in parallel by checking each sequence segment on a 


separate core. 
by appending the data (ora 


hash of some data) i . The recording of the state, 
index and data as it was appended into the sequences provides a timestamp 
that can guarantee that the data was created sometime before the next hash 
was)generateduimithersequence. This design also supportsthorizontal scaling 
as multipleygeneratorsycanysyuchronize amongst each other by mixing their 


state into each others’ sequences. Horizontal scaling is discussed in depth in 
Section 4.4 


4.1 Description 


The system is designed to work as follows. With a cryptographic hash func ~ 
tion, whose output cannot be predicted without running the function (e.g. 
sha256, ripemd, etc.), run the function from some random starting value 
and take its output and pass it as the input into the same function again. 
Record the number of times the function has been called and the output at 
each call) The starting random value chosen could be any string, like the 
headline of the New York times for the day. 


For example: 


PoH Sequence 


Index Operation Output Hash 
1 sha256(”any random starting value”) hash1 
2 sha256(hash1) hash2 
3 sha256(hash2) hash3 


Where hashN represents the actual hash output. 


It is only necessary to publish a subset of the hashes and indices at an 
interval. 


For example: 


PoH Sequence 


Index Operation Output Hash 
1 sha256("any random starting value”) hashi 
200 sha256(hash199) hash200 
300 sha256(hash299) hash300 


As long as the hash function chosen is collision resistant, this set of hashes 
can only be computed in sequence by a single computer thread. This follows 


from the fact that there is no way to predict what the hash value at index 
300 is going to be without actually running the algorithm from the starting 


value 300 times. Thus wecan thus inier fonmi the data structure that real” 
time has passed between index 0 and index 300. 

In the example in Figure 2, hash 62£51643c1 was produced on count 
510144806912 and hash c43d862d88 was produced on count 510146904064. 
Following the previously discussed properties of the PoH algorithm, we can 
trust that real time passed between count 510144806912 and count 510146904064. 


4.2 Timestamp for Events 


This sequence of hashes can also be used to record that some piece of data 
wasicreated)beforevayparticular hashyindexywasjgenerated. Using a ‘combine’ 


function to combine the piece of data with the current hash at the current 


index. The data can simply be a cryptographically unique hash of arbitrary 
event data. The combine function can be a simple append of data, or any 
r The next generated hash represents a 
timestamp of the data, because it could have only been generated after that 
specific piece of data was inserted. 


For example: 


PoH Sequence 


Index Operation Output Hash 
1 sha256(”any random starting value”) hash1 
200 sha256(hash199) hash200 
300 sha256(hash299) hash300 


hash 62f£51643cl... 
count 510144806912 


hash 3d039ef62e... 


count $10145855488 


hash c43d862d88... 
count 510146904064 


Figure 2: Proof of History sequence 


Some external event occurs, like a photograph was taken, or any arbitrary 
digital data was created: 


PoH Sequence With Data 


Index Operation Output Hash 
1 sha256("any random starting value”) hasht 
200 sha256(hash199) hash200 
300 sha256(hash299) hash300 
336 sha256(append(hash335, photograph_sha256)) hash336 


Hash336 is computed from the appended binary data of hash335 and the 
sha256 of the photograph. The index and the sha256 of the photograph are 
recorded as part of the sequence output. So anyone verifying this sequence 
can then recreate this change to the sequence. The verifying can still be done 
in parallel and it’s discussed in Section 4.3 


POH Sequence 


Index Operation Output Hash 
1 sha256("any random starting value”) hash1 
200 sha256(hash199) hash200 
300 sha256(hash299) hash300 
336 sha256(append(hash335, photograph1__sha256)) hash336 
400 sha256(hash399) hash400 
500 sha256(hash499) hash500 
600 sha256(append(hash599, photograph2__sha256)) hash600 
700 sha256(hash699) hash700 


Table 1: PoH Sequence With 2 Events 


Because the initial process is still sequential, we can then tell that things 
entered into the sequence must have occurred sometime before the future 
hashed value was computed. 


In the sequence represented by Table 1, photograph2 was created before 
hash600, and photograph1 was created before hash336. Inserting the data 
into the sequence of hashes results in a change to all subsequent values in the 
sequence. As long as the hash function used is collision resistant, and the 
data was appended, it should be computationally impossible to pre-compute 
any future sequences based on prior knowledge of what data will be inte- 
grated into the sequence. 


The data that is mixed into the sequence can be the raw data itself, or 
just a hash of the data with accompanying metadata. 


In the example in Figure 3, input cfd40df8... was inserted into the Proof 
of History sequence. The count at which it was inserted is 510145855488 and 
the state at which it was inserted is 3d039eef3. All the future generated 
hashes are modified by this change to the sequence, this change is indicated 
by the color change in the figure. 
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hash 62f5163cl1... 
count 510144806912 


hash 3d039eef3... 
count 510145855488 
input cfd4f0df8... 


input cfd40df8... 


hash 79a5dfldc... 
count 510146904064 


Figure 3: Inserting data into Proof of History 


4.3 Verification 


The sequence can be verified correct by a multicore computer in significantly 
less time than it took to generate it. 


For example: 


Core 1 
Index Data Output Hash 
200 sha256(hash199) hash200 
300 sha256(hash299) hash300 
Core 2 
“Index Data Output Hash 
300 sha256(hash299) hash300 
400  sha256(hash399) hash400 


Given some number of cores, like a modern GPU with 4000 cores, the 
verifier can split up the sequence of hashes and their indexes into 4000 slices, 
and in parallel make sure that each slice is correct from the starting hash to 
the last hash in the slice. If the expected time to produce the sequence is 
going to be: 


hash 3d039eef6... 
count 510145855488 
input cfd40dzf8s... 


Verifier 
core 1 


Figure 4: Verification using multiple cores 


hash 62£5164cl... 
count 510144806912 


hash 79a5aaldc... 
count 510146904064 


K= 


Verifier 
core 2 


Total number of hashes 
Hashes per second for 1 core 


The expected time to verify that the sequence is correct is going to be: 


Total number of hashes 
(Hashes per second per core * Number of cores available to verify) 


In the example in Figure 4, each core is able to verify each slice of the 
sequence in parallel. Since all input strings are recorded into the output, with 
the counter and state that they are appended to, the verifiers can replicate 
each slice in parallel. The red colored hashes indicate that the sequence was 
modified by a data insertion. 


4.4 Horizontal Scaling 


It’s possible to synchronize multiple Proof of History generators by mixing 
the sequence state from each generator to each other generator, and thus 
achieve horizontal scaling of the Proof of History generator. This scaling 
is done without sharding. The output of both generators is necessary to 
reconstruct the full order of events in the system. 


PoH Generator A PoH Generator B 
Index Hash Data || Index Hash Data 
hashia hashib 
hash2a hashib hash2b hashia 


hash3a hash3b 
hash4a hash4b 


Given generators A and B, A receives a data packet from B (hash1b), 
which contains the last state from Generator B, and the last state generator 
B observed from Generator A. The next state hash in Generator A then de- 
pends on the state from Generator B, so we can derive that hashlb happened 
sometime before hash3a. This property can be transitive, so if three gener- 
ators are synchronized through a single common generator A © B © C, 
we can trace the dependency between A and C even though they were not 
synchronized directly. 


In Figure 5, the two generators insert each other’s output state and record 
the operation. The color change indicates that data from the peer had mod- 
ified the sequence. The generated hashes that are mixed into each stream 
are highlighted in bold. 

The synchronization is transitive. A + B + C There is a provable order 
of events between A and C through B. 

Cni i Anini 
tions with availability of 0.999 would have 0.999'° = 0.99 availability. | 


4.5 Consistency 


Usersyareyexpectedstoybeyablestoyenforceyconsistency of the generated se- 
quence and make it resistant to attacks bysinserting the last observed output 
of the sequence they consider valid into their input. 
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hash 62f5164cl... 
count 510144806912 


hash 3d039eef... 
count 510145855488 
input 2z£d23423... 


hash 79a5idzc... 
count 510146904064 


Figure 5: Two generators synchronizing 


hash 99ada5ldc... 
count 310146904064 


hash b3a83512... 
count 310146904064 
input 79a5ldzc... 


PoH Sequence A PoH Hidden Sequence B 
Index Data Output Hash || Index Data Output Hash 
10 hashi0a |} 10 hashi0b 


20 Event1l hash20a || 20 Event3 hash20b 
30 Event2 hash30a || 30 Event2 hash30b 
40 Event3 hash40a || 40 Event1 hash40b 


A malicious PoH generator could produce a second hidden sequence with 
the events in reverse order, if it has access to all the events at once, or is able 
to generate a faster hidden sequence. 


To prevent this attack, eachiclient#generated Event should contain within 
itself the latest hash that the client observed from what it considers to be 
ence. So when a client creates the ”Event1” data, they should 

append the last hash they have observed. 
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PoH Sequence A 
Index Data Output Hash 
10 hashi0a 
20 Eventl = append(eventl data, hash10a) hash20a 
30 Event2 = append(event2 data, hash20a) hash30a 
40 Event3 = append(event3 data, hash30a) hash40a 


When the sequence is published, Event3 would be referencing hash30a, 
and if it’s not in the sequence prior to this Event, the consumers of the 


sequence know that it’s an invalid sequence. “The partial reordering attack” 


observed.an event and when the event was entered. Clients should then be 

; , i 
bietemitesolmarethatioegsneingsumesthessdemiyroneemionthesho a hel — mi. 

To prevent a malicious PoH generator from rewriting the client Event 


hashes, the clients can submit a signature of the event data and the last 
i i bof iiai 


PoH Sequence A 


Index Data Output Hash 

10 hash10a 
Event1 = sign(append(event1 data, hash10a), 

a Client Private Key) hastala 
Event2 = sign(append(event2 data, hash20a), 

ao Client Private Key) pba 

40 Event3 = sign(append(event3 data, hash30a), hash40a 


Client Private Key) 


Verification of this data requires a signature verification, and a lookup of- 
ERAT oie hi 


Verify: 

(Signature, PublicKey, hash30a, event3 data) = Event3 
Verify(Signature, PublicKey, Event3) 

Lookup(hash30a, PoHSequence) 


In Figure 6, the user-supplied input is dependent on hash Oxdeadbeef... 
existing in the generated sequence sometime before it’s inserted. The blue 
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hash Oxdeadbeef... 
count 510144806912 
hash OxbadcOde... 

count 510145855488 
input OxcOdebab... 
hash 79aSldzc... 

count 510146904064 


Figure 6: Input with a back reference. 


input: event data 
last hash: Oxdeadbeef. . 
event hash: OxcOdebab... 


User event hash OxcOdebab is generated 
with the hash Oxdeadbeef. It links 
Oxdeadbeef to the next generated block 
Oxbadc0de. Which prevents a malicious 
Loom from reordering the events in a 
hidden side sequence. 


top left arrow indicates that the client is referencing a previously produced 
hash. The client’s message is only valid in a sequence that contains the hash 
Oxdeadbeef.... The red color in the sequence indicates that the sequence has 
been modified by the clients data. 


4.6 Overhead 


4000 hashes per second would generate an additional 160 kilobytes of data, 
and would require access to a GPU with 4000 cores and roughly 0.25-0.75 
milliseconds of time to verify. 


4.7 Attacks 
4.7.1 Reversal 


‘sequence after the second event. This delay should allow any non malicious 
m l : } ' areren 
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4.7.2 Speed 


Having multiple generators may mMakeldeploymentymorewesistant to attacks. 
One generator could be high bandwidth, and receive many events to mix 


4.7.3 Long Range Attacks 


Long range attacks involve acquiring old discarded client Private Keys, and 
generating a falsifed ledger |10]. Proof of History provides some protection 
against long range attacks. Almalicioustuser (that) gains\access\tovoldprivate 
‘keys would have to recreate a historical record that takes as much time as the 


original one they are trying to forge. This i O 
processor)than the network is currently using, 
Gever catchmuplinihistory length. 

Additionally, a single source of time allows for construction of a simpler 
Proof of Replication (more on that in Section 6). Since the network is de- 
signed so that all participants in the network will rely on a single historical 
record of events. 

PoRep and PoH together should provide a defense of both space and time 
against a forged ledger. 


5 Proof of Stake Consensus 


5.1 Description 


This specific instance of Proof of Stake is designed for quick confirmati ; 
the current sequence produced by the Proof of History generator, for voting 
cbehayingwalidatorsssThis algorithm d n 


14 


5.2 Terminology 


bonds Bonds are equivalent to a capital expense in Proof of Work. A miner 
buys hardware and electricity, and commits it to a single branch in a 


Proof of Work blockchain. A vonds coin that the validator commits 
as collateral while they are validating transactions. 


slashing The proposed solution to the nothing at stake problem in Proof 
of Stake systems [7]. Whena proof of voting for a different branch is 
published, that branch can destroy the validator’s bond. This is an 
economic incentive designed to discourage validators from confirming 
multiple branches 


super majority A super majority is 
A super majority vote indicates that the network has reached 
consensus, and 


maliciously for this’ branchito'beinvalid. This would put the economic 


cost of an attack at ard of the market cap of the coin. 


5.3 Bonding 


A bonding transaction takes a amount of coin and moves it to a bonding 
account under the user’s identity. (Goinstimithe:bondingyaccountycannotybe 
‘spent and have to remain in the account until the user removes them. The | 
(ini enannoeatoeneoina oteenealaencep cenit 


5.4 Voting 


It is anticipated that the Proof of History generator will be able to publish 
a signature of the state at a predefined period. Each bonded identity must 
confirm that signature by publishing their own signed signature of the state. 
The vote is a simple yes vote, without a no. If super majority of the bonded 
identities have voted within a timeout, then this branch would be accepted 
as valid. 
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5.5 Unbonding 

Missing N ksotl : _— tisilstestor 
N is a dynamic value based on the fatio of stale to active votes. N 

increases as the number of stale votes increases. man event of a largo 


network partition, this allows the larger branch to recover faster than the 
smaller branch. — 


5.6 Elections 


Election for a new PoH generator occur when the PoH generator failure is 
detected. The validator with the largest voting power, or highest public key 
addressiifitheretisyaytiedsypicked as the new PoH generator. 

A ES E 

a 

To switch votes, a validator needs to vote at a higher PoH sequence 
both ane : itel 
T r EEA T j . Vote switching is expected to 
be designed so that it can only occur at a height that does not have a super 
majority. 
Once a PoH generator is established, aGeeondary can be elected to take 
over the transactional processing duties. If a Secondary exists, it will be 


The petin is designed so that the Secondary becomes Primaryrand» 


5.7 Election Triggers 
5.7.1 Forked Proof of History generator 


PoH generators are designed with an identity that signs the generated se- 
quence. A fork can only occur in case the PoH generator’s identity has been- 


compromised. A fork is detected because twordifferentyhistoricalrecordsjhave 
been published on the same PoH identity. — 


5.7.2 Runtime Exceptions 


A hardware failure or a bug, or a intentional error in the PoH generator 
could cause it to generate anjinvalidjstatevand publishva\signaturesofithep 
the correct signature via gossip and this event would trigger a new round of- 


5.7.3 Network Timeouts 
A network timeout would trigger an election. 


5.8 Slashing 


Slashing occurs when a validator votes two separate sequences. A proof of 
malicious vote will removerthe bonded sins from circulation and add them 


A vote that includes a previous vote on a contending sequence is not 
eligible as proof of malicious voting. Instead of slashing the bonds, this vote 
removes the currently cast vote on the contending sequence. 

Slashing also occurs ia votes cast fomanunvalidlhash generated by the 


PoH generator. The générater is expected to randomly generate an invalid 
state, which would trigger a fallback to Secondary. 


5.9 Secondary Elections 


Secondarywandwlower ranked Proof of History generators camybeyproposed 
and approved. A proposal is cast on the primary generator’s sequence. The 
proposal contains a timeout, if the motion is approved by a super majority 
of the vote before the timeout, the Secondary is considered elected, and will 
take over duties as scheduled. Primary can do a Soft handover to Secondary 
by inserting a message into the generated sequence indicating that a handover 
will occur, ominserting anm invalid stato and forcing the network to fallback 
to Secondary. 

If a Secondary is elected, and the primary fails, the Secondary will be 
considered as the first fallback during an election. 
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5.10 Availability 


CAP systems that deal with partitions have to pick Consistency or Avail- 
ability. Our approach eventually picks“Availability, but becauseswe have” 
n objective measure of time, Consistency is picked with reasonable human 


a 
timeouts. 

Proof of Stake verifiers lock up some amount of coin in a “stake”, which 
allows them to vote for a particularisetiof transactions, Locking up coin is a 
transaction that is entered into a PoH stream, just like any other transaction. 

“PoWotep a PoS verifier has to sign the hash of thestate, as it was computed 
after processing all the transactions to a specific position in the PoH ledger. 
This vote is also entered as a transaction into the PoH stream. Looking at 


the PoH ledger, we can then énferyhowsmuchstime passed! betweemeachvote; 


and i 
To deal with partitions with reasonable human timeframes, we propose 


ae o. Ina largé partition, like a partition 
that is missing 5 t or more of the verifiers, thesunstakingyprocessiisiveryivery> 


The diane. in (nie for a Heak isi regain A liveness allows us as cuchomers 


of the network human timeframes to pickalpartition thatwe wantitorcontinue 


5.11 Recovery 


In the system we propose, thededgerican be fully recoveredifromanyfailure. 
That means, anyone in the world can pick any random spot in the ledger and 
create a valid fork by appending newly generated hashes and transactions. 
If all the verifiers are missing from this fork, it would take a very very long 


R rity consensus. So full recovery with zero available validators 
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would require a very large amount of hashes to be appended to the ledger, 


and only after all the unavailable validators have been unstaked will any new | 


5.12 Finality 


PoH allows verifiers of the network to observe what happened in the past with 
some degree of certainty of the time of those events. As the PoH generator is 
producing a stream of messages, all the verifiers are required to submit their 
signatures of the state within 500ms. This number can be reduced further 


depending on network conditions. Since each verification is entered into the 


5.13 Attacks 
5.13.1 Tragedy of Commons 


The PoS verifiers simply confirm the state hash generated by the PoH gen- 
erator. There is an economic incentive for them to do no work and simply 
approve every generated state hash. To avoid this condition, the|PoH gener- 


ator should inject an invalid hash at a random interval. Any voters for this 
hash should be slashed? When the hash is generated, the network should 
immediately promoteithe|Secondaryielected)PoH generator. 

Each verifier is required to respond within a small timeout - 500ms for 
example. The timeout should be set low enough thata malicious verifier has 


a S 


into the stream fast enough. 


5.13.2 Collusion with the PoH generator 


‘when the invalid hash is going to be produced and not vote for it. This 


scenario is really no different than the PoH identity having a larger verifier 
stake. The PoH generator still has to do all the work to produce the state 
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5.13.3 Censorship 


Censorship or denial of service could occur 


i i The protocol can defend 
against this form of attack by d 


stale. In the event of a denial of service, the larger partition will be designed» 
to fork and censor the Byzantine bond holders. The larger network will re- 


iaiaaeaia. The smaller Byzantine 
partition would not be able to move forward for a longer period of time. 


The algorithm would work as follows. Awmajorityroftthemetworkywould™ 
elect a new Leader. The Leader would then censor the Byzantine bond 
icipating. Proof of History generator would have to continue 
i , , , 


bonds have become stale so the bigger network has a super majority. The rate 


at which bonds become stale would be dynamically based on what percentage 
of bonds are active. So the Byzantine minority fork of the network would have 
to wait much longer than the e mipya fork to recover a super majority. Once ~ 


5.13.4 Long Range Attacks 


PoH provides a natural defense against long range attacks. Recovering the 


ledger from any point in the past would require theyattackertovovertakesthe 
valid ledger in time by outpacing the speed of the PoH generator. 
The consensus protocol provides a second layer of defense, as any attack 


would have to take longer than the time it takes to unstake all the valid 
Walidators. It also creates an availability “gap” in the history of the ledger. 
5.13.5 ASIC Attacks 


Two opportunities for ASIC attacks exist in this protocol - during partition, 
and cheating timeouts in Finality. 


For ASIC attacks during)Partitions, theRateatwhich*bondsareunstakeds 
is non-linear, and for networks with large partitions the rate is orders of | 
“magnitude slower than expected gains from an ASIC attack. 
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For ASIC attacks during Finality, the vulnerability allows for ge 


r. The PoH 
generator can then use its faster ASIC to generate 500ms worth of hashes 
in less time, and allow for network communication between PoH paan ai i 
and the collaborating nodes. But, 
the ton yh ani ta oe omnes 

the exact counter when they expect to insert the failure. This scenario is no 
different than a PoH generator and all the collaborators sharing the same 
identity vs. having a single combined stake and only using 1 set of hardware. 


6 Streaming Proof of Replication 


6.1 Description 


Filecoin proposed a version of Proof of Replication [6]. The goal of this 
version is to have fast and streaming verifications of Proof of Replication, 
which are enabled by keeping track of time in a Proof of History generated 


sequence. Replicationlismotiusedjasjajconsensusyalgorithm, but is a useful 


tool to account for the cost of storing the blockchain history or state at a 
high availability. 


6.2 Algorithm 


As shown in Figure 7 CBC encryption encrypts each block of data in se- 

quence, using the previously encrypted block to XOR the input data. 

generated via a Proof of History sequence. This ties the key to a replicator’s 
ity, and to a specifi’ Proof of History sequence? Only specific hashes 


can be selected. (See Section 6.5 on Hash Selection) 


The QgjgabeiniGelullpenonspiedsbloaabysbles:. Then tameme. 
the key is used to seed a pseudorandom number generator that selects a 
random 32 bytes from each block. 


è Current block to be encrypted 
èe Previous encrypted block data to be XORd with the current block before encryption 
e Ensures that the encryption can only be done sequentially, and it can be streamed 


Figure 7: Sequential CBC encryption 


of. Again, the hash is signed, and random slices are ~ 


6.3 Verification 


With N cores, each core can stream encryption for each identity. Total space 
required is 2blocks x Ncores, since the previous encrypted block is necessary 
to generate the next one. Each core can then be used to generate all the 
proofs that were derived from the current encrypted block. 


“encrypts The proofs themselves consume few random bytes from the block, 
so the amount of data to hash is significantly lower than the encrypted block 
size. The number of replication identities that can be verified at the same 
time is equal to the number of available cores. Modern GPUs have 3500+ 


cores available to them, albeit at 5-3 the clock speed of a CPU. 
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e random bytes selected 
from the block 
e Proof of History hash 


Figure 8: Fast Proof of Replication 


6.4 Key Rotation 


6.5 Hash Selection 


The Proof of History generator publishes a hash toybeusedybystherentiremet=yp 
work for encrypting Proofs of Replication, and for using as the pseudorandom 
TSE anr e f 

The hash is publi 1 
thestimesitytakesytoyencryptythesdatayset. Each replication identity must use 
the same hash, and use the signed result of the hash as the seed for byte 
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selection, or the encryption key. 


ag 


ashstorgenerateranspecific hash. This attack is discussed more in 5.13.2. 


6.6 Proof Validation 


The is not expected to validate the submitted Proof of 
Replication proofs. It ee 


verified proofs submitted by the replicator’s identity. A proof is expected to 
of the validators in the network. 

The verifications are collected by the replicator via p2p gossip network, 
and submitted as one packet that contains a super majority of the validators 
in the network. This packet verifies all the proofs prior to a specific hash gen- 
erated by the Proof of History sequence, and camcontainmultiple repliéator 
identities at once. 


6.7 Attacks 
6.7.1 Spam 


A malicious user could create many replicator identities and spam the net- 
work with bad proofs. To facilitate faster verification, nodes are required 
t . f 

on. 

The Proof of Replication that is designed in this paper allows for cheap 
verification of any additional proofs, as they take no additional space. But 
each identity would consume 1 core of encryption time. The replication target- 
should be set to a maximum size of readily available cores. Modern GPUs 


ship with 3500+ cores. 


6.7.2 Partial Erasure 


A replicator node could attempt to i j 
cisiiinibeeniinete'e. Then e 


For example, a user storing 1 terabyte of data erases a single byte from 
each 1 megabyte block. A single proof that samples 1 byte out of every 
megabyte would have a likelihood of collision with any erased byte 1 — (1 — 
1/1, 000, 0000)1:009-° — 0.63. A ftermSuproofsthallikelihood is 099. 


6.7.3 Collusion with PoH generator 


Thisyattackscouldsonlyshenefitvaysingleweplicatorsidentity. Since all the 
identities have to use the same exact hash that is cryptographically signed 
with ECDSA (or equivalent), the resulting signature is unique for each repli- 
cator identity, and collision resistant. 


6.7.4 Denial of Service 


This creates an opportunity fora denial of servicelattack on the network 


by creating a large number of valid replicator identities. 


To limit this attack, thesconsensus"protocolichosen forthemetwork Can 
select a replication target, and award the replication proofs that meet the de- 
= istics, ike availabi i Á 


etc... 


6.7.5 Tragedy of Commons 
The PoS verifiers could simply confirm PoRep without doing any work. The 
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Verifier 


-7A Verifier 


e Generator: 
Proof of 
History 

e Verifier: Proof 
of Replication 
e Inbound user 
transactions 

e Stripped traffic 
from Loom to 


Generator 


Spools 

e P2P Verifier 
traffic 

e Replicated 
blockchain 
broadcast 


Figure 9: System Architecture 


To further avoid this scenario, the P6Replvérifiersican!submitefalseyproofs 
. They can prove the proof is false by providing 
. Any PoS verifier that confirmed 


t 


a false proof would be slashed. 


7 System Architecture 


7.1 Components 


7.1.1 Leader, Proof of History generator 


The Leader is an elected Proof of History generator. It consumes arbitrary 
user tramsactionsyandyoutputsya Proof of History sequence of all the transac- 
tions thatyguaranteeyayuniqueyglobalyorder in the system. After each batch 
of transactions the Leader outputs a signature of the state that is the result 


. This signature is signed with the 
identity of the Leader. 
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7.1.2 State 


The statelis a naive hashstablesindexedibysthemser’syaddress. Each cellicon= 
‘aims the full user’s address and the memory required for this computation. 


For example, SN, 
the Transaction table contains: N 7 
159 191 223 


0 31 63 95 127 * 255 


Ripemd of Users Public Key 


for a total of 32 bytes. 
The Proof of Stake bond’s table contains: 


0 31 63 95 127 159 191 22 
Ripemd of Users Public Key 
Last Vote 


3 255 


for a total of 64 bytes. 


7.1.3 Verifier, State Replication 
The Veri 


erifier nodes replicate and provide high availability of the blockchain 
State. The replication target isselectedibyatherconsensus algorithm, and the 
validators in the consensus algorithm sélect and vote the Proof of Replication 
nodes they approve of based on off-chain defined criteria. 
The network could be configured with a minimum Proof of Stake bond 
size, and a requirement for a single replicator identity per bond. 


7.1.4 Validators 
These nodes are consuming bandwidth from Verifiers. They are virtual nodes, 


and can run on the same machines as the Verifiers or the Leader, or on 
separate machines that are specific to the consensus algorithm configured for 
this network. 


7.2 Network Limits 


The leader is expected to be able to take incoming user packets, order them 
in the most efficient way possible, and sequence them into a Proof of History 


27 


e Inbound user transactions 


e Outgoing traffic to Verifiers 
Generator 


Figure 10: Generator network limits 


sequence that is published to downstream Verifiers. Efficiency is based on 


memory access patterns of the transactions, so the transactionsyareyordered 
to minimize faults and to maximize prefetching. — 


ee a format: 


223 255 


Last Valid Hash cee 


Signature 1 of 2 
Signature 2 of 2 


Size 20+ 8 + 16 + 8 + 32 + 32 + 32 = 148 bytes. 


The minimal payload that can be supported would be 1 destination ac- 
count with a minimum size of 176 bytes. 


With payload: 
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Signature 1 of 2 


Signed 


Signature 2 of 2 


With payload the minimum size: 176 bytes 


The Proof of History sequence packet contains the current hash, counter, 
and the hash of all the new messages added to the PoH sequence and the 
state signature after processing all the messages. This packet is sent once 

-every N messages are broadcast. 


Proof of History packet: 
0 31 63 95 127 15 


Signature 1 of 2 


Signed 


Signature 2 of 2 


Minimum size of the output packet is: 132 bytes 


On a lgbps network connection the maximum number of transactions 
possible is 1 gigabit per second / 176 bytes = 710k tps max. Some loss 
(1 — 4%) is expected due to Ethernet framing. The spareycapacity over 
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the target amount for the network cán be used to increase availability by 
p and striping it to the available 


7.3 Computational Limits 
Œachitransactionmequiresrandigestøverificatiom This operation does not use 


any memory outside of the transaction message itself and can be parallelized 
independently. Thus throughput is expected to be limited by the number of 
cores available on the system. 

GPU based ECDSA verification servers have had experimental results of 
900k operations per second [9]. 


7.4 Memory Limits 


A naive implementation of the state as a 50% full hashtable with 32 byte en- 
tries for each account, would theoretically fit 10 billion accounts into 640GB. 
Steady state random access to this table is measured at 1.1 * 10’ writes or 
reads per second. Based on 2 reads and two writes per transaction, memory 
throughput can handle 2.75m transactions per second. This was measured 
on an Amazon Web Services 1TB x1.16xlarge instance. 


7.5 High Performance Smart Contracts 


Smart contracts are a generalized form of transactions. These are programs) 


that run'on eachmodeandmmodifysthesstate, This design leverages extended 
BerkeleysPacket)Piltersbytecode, which is fast and easy to analyze, and JID) 
bytecode as the smart contracts language. | 


One of its main advantages is a zero cost Foreign Function Interface. 


Intrinsics, or functions that are implemented on the platform directly, are | 
a Calling the intrinsics suspends that program and 
schedules the intrinsic on a high performance server. Intrinsics are batched 
together to execute in parallel on the GPU. 

In the above example, two different user programs call the same intrinsic. 
Each program is suspended until the batch execution of the intrinsics is 


complete. An example intrinsic is ECDSA verification. Batching these calls 
to execute on the GPU can increase throughput by thousands of times. 
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e suspend 
s e resume 
a T 


Intrinsic A Intrinsic A 
| GPU Intrinsic | 


Server 


Figure 11: Executing BPF programs. 


This trampoline require 
since the BPF bytecode has a well defined context for all the memory that 
it is using. 

eBPF backend has been included in LLVM since 2015, so anys LLVM 
It’s been in the 
Linux kernel since 2015, and the first iterations of the bytecode have been 
around since 1992. A single pass can check eBPF for correctness, ascertain 
its runtime and memory requirements and convert it to x86 instructions. 
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